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ABSTRACT 



The European Training Foundation's Advisory Forum 
established four working groups, one of which was to conduct activities 
relating to new National Vocational Education and Training (VET) Standards. 
The 1995 working session accepted definitions of VET Standards and described 
three models. The 1996 working session decided on a checklist of steps to 
develop a standard. The 1997 working session worked out strategies for 
implementation of standards. The objective for 1998 was to develop an 
empirical evaluation/progress review of pilot projects for VET Standards. A 
questionnaire was circulated among members of the subgroup to evaluate t'he 
procedure of standard development and implementation in member and partner 
countries. It followed results of the group's work starting with models of 
standards, development steps, and implementation procedure. Sixteen partner 
and European Union Member States responded. Findings indicated the following: 
school -based VET systems were most prevalent; they combined classroom 
teaching with demonstrations and performance of work tasks in workshops and 
laboratories; the traditional VET Standard model was the occupation model; 
the ministry of education took the initiative for VET Standard setting in 
two-thirds of the states; and all countries referred to the five categories 
of VET Standards the group defined. Current activities in the Russian 
Federation, Ukraine, Macedonia, and Germany were examined. Ways to measure 
labor market needs were identified. (YLB) 
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1. New Vocational Education and Training (VET) - 
Standards - an international agenda 

1.1. What are Vocational Education and Training Standards? 



New National VET-Standards have become the education sector’s global response to 
fundamental changes. Considering EU-partner states, the countries of the former “Eastern 
Bloc”, and EU-member states, the dimension of these changes differs widely. While partner 
states have to transform their VET-systems as a whole, EU-member countries “only” have to 
modernise their systems, to review their standards and develop new standards for emerging 
labour markets. Nevertheless, there is one common problem: How to set new VET-standards 
that are led by the needs of the young generation and the needs of employers and modem 
workplaces as well? 

In the international discussions there is a lot of misunderstanding about the term “standard”. In 
order to achieve a common understanding subgroup C “VET-standards” uses the ETF-glossary 
which outlines on “ standard of education” : 

“The extent to which the education system meets what is expected of it. This may be in 

terms of quality of teaching, the content of learning Vocational education standards tend 

to be used as the basis for curriculum design and education and training delivery. 
Occupational standards tend to be used to specify what is required of people at work. 
Increasingly the distinction between the two is being eroded as standards are developed to 
integrate education and training outputs with the requirements of the world of work. 
Either may be used as qualification standard.” 

VET-standards in sub group C’s understanding “apply to a wide set of work roles across the 
economy. They are broadly based and they represent measures of the extent to which an 
individual can meet the demands of performance. National Vocational Education Standards 
are those set for all practitioners within a specific occupation in a country” (ETF-glossary). 



1.2. What challenges are the reasons to develop new VET-Standards? 



The impact of information and communication technologies on work organisations and private 
life, innovations in the delivery of services, the change in the international framework of 
national economies onto the fundamental changes of the political and economic systems of the 
transformation states in central and eastern Europe and the former Soviet Union has urged all 
countries to revise their VET-standards or occupational classifications or develop a new system 
of occupations and standards. 
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What used to be a frequent national procedure of reviewing and modernising VET curricula, a 
task for educators and restricted to their field of influence, has now become a global endeavour 
with economists, labour market specialists, employers, trade unions, international organisations 
and researchers involved. The already mentioned transformation processes in the former 
eastern bloc, the unification of Europe and the North American Free Trade Agreement 
(NAFTA) challenge the flexibility and mobility of the workforce together with changing 
requirements in human capabilities like communication skills, customer orientation, qualitative 
performance and delivery. 

Within companies the loss of hierarchic layers through „lean“ re-engineering, the demand for 
more efficiency and cost cutting called for more and better education and training which were 
not to be met by the old narrow occupations or fragmented jobs. 

The new set of requirements for skilled workers or employees goes far beyond the average skill 
profiles in a taylorised work organisation: language (communication skills) and mathematics 
(use of numbers) have become important occupational skills for the majority of occupations 
because they are the prerequisite to master the common tool across the workforce: the 
computer. A broader intellectual base for VET is required since the skilled worker or employee 
is expected to plan, carry out and control his/her work tasks autonomously. It has to be stated 
here, that the perception of VET in the former eastern bloc states has always been emphasising 
much more on general education within VET occupations than it was the rule in western 
countries. This advantage can be used for a better and new understanding of standard setting 
with vocational and general education elements. 

The dynamic of change has forced employers to recognise that their work force’s ability to 
adapt to changes and to master new tasks and challenges results from a broad theoretical and 
practical basic and recurrent education. They attach much more value to a worker’s ability to 
learn how to learn than the mastery of traditional skills. 

In the actual debate on the reform of VET in Europe, east and west, the problem of a mutual 
recognition of VET certificates has been playing a major role. This discussion had begun in the 
early eighties within the European Community but soon was brought to an end when member 
states agreed that only regulated professions were to be recognised. This only concerned 
academic professions. Given the very different VET systems in Europe the attempt to develop a 
common classification of European VET Standards proved to be impossible. The very different 
VET-systems did not allow mutual recognition within very different economic and educational 
contexts. Beginning in 1985 CEDEFOP has managed within a decade, together with tripartite 
expert groups from all member states, to compare the different occupational classifications and 
occupational profiles of the EU countries to a large extent. This was an important progress for 
VET development in Europe because there is now much more transparency and knowledge 
about VET standards and occupational classifications. But this work lead to the conclusion not 
to grant the recognition of certificates on skilled workers' level in Europe. Bilateral recognition 
for a number of occupations as between France and Germany or Germany and Austria remain 
exceptions from the rule 

In order to establish a network of VET information like CEDEFOP has developed for EU 
member states ETF has helped central and eastern European countries and the New 
Independent States to create observatories which — among other duties — collect data to 
identify criteria for converging curriculum and standard development work with the Union. 



Thus, a network of co-operating national partners could help to develop a comparative 
methodology to validate standards. 

The redesigning of VET systems and occupational standards is obviously increasing the status 
of VET and enhancing parity of esteem. The relationship of general and vocational education 
and training is widely discussed, since the definition of „general“ education and „basic 
vocational education” has become more difficult than before because the boundaries are 
blurring. The conceptual and theoretical content of VET is growing since knowledge is soon 
becoming obsolete. Therefore, one of the most difficult tasks of curriculum development and 
standard setting in VET is the definition of the basic skills and knowledge or the fundament for 
an occupational family i.e. technical occupations, business, health care. 

In most EU-partner countries governments still play the decisive role as it comes to the 
definition of labour market needs, to define, develop and implement new VET standards and to 
break up old classifications. The role of employers and trade unions as government partners in 
planning and implementing modern VET standards is yet to be defined and to be developed in 
those countries. In contrary, EU- countries like the Scandinavian nations, Benelux, Ireland, 
Austria and Germany have a more or less developed research based tripartite preparation and 
decision making process of standard setting which is equally to be developed in Greece, Spain 
and Portugal. The UK's VET- system is led by industry (with a small input from unions) which 
proposes measurable benchmarks (standards) of performance in one's job. They are called 
“occupational standards” because they are components of occupations as well. The central and 
eastern European countries still lack an involvement of organised labour and employer 
organisations. Other countries which had no VET-standards in the past have created new 
institutions like the National Skill Standards Board which has been established in the USA in 
1994 serving as a standard development association. We are witnessing a world-wide move 
towards modern VET Standards which are the backbones of new emerging systems. 




7 



2. ETF-Advisory Forum and Subgroup work 



When the European Training Foundation (ETF) was established it’s Advisory Forum became 
the bottom-up-advice-giving-body. Representatives of all EU-member states, the partner states, 
social partners and international agencies were given the task to define the needs of the 
transformation states in the field of VET and management training and advise ETF with it’s 
future work programmes. 

On the first plenary meeting of the Advisory Forum in October 1994 it was decided to establish 
four working groups. The Forum defined four fields of activities, that should be discussed in 
subgroups A-C: 

VET-Financing and decentralisation (A), the role of the social partners in VET-planning (B), 
VET-Standards (C), the roles of key players in continuing training (D). Subgroup C was given 
the topic „Standards in vocational education and training" which became the most important 
political VET-issue since 1994 in the partner countries and EU-member states as well. 



3. Subgroup C „VET-Standards“ procedure 1995 - 1997 

FROM „DEFINITION“ TO IMPLEMENTATION" 

Three working sessions on the topic VET- Standards were held in Subgroup C dealing with 

• principles and definition of VET-standards (1995); 

• VET-standard development (1996); 

• VET-standard implementation (1997). 

In order to ensure continuity for the Subgroup’s further discussion with new members and to 
save time recalling earlier definitions and recommendations here is an abstract of the 
Subgroups most important outcomes together with short comments on alterations and 
amendments that have been added within the course of work: 

3.1. Principles and definitions of VET- standards (1995) 

It is interesting to follow — in the light of the discussion since 1995 — the definitions that 
were found for VET-standards. The starting basic definition read as follows 

VET-Standards include the general description of work tasks to be carried out within the 
context of the profession in question, as well as an outline of relevant skills and 
qualifications. 

At the end of an intensive discussion the following definition was accepted by the Subgroup for 
further work: 

VET-Standards are general descriptions of work tasks, which have to be practised within 
the framework of the relevant profession, as well as the presentation of the appropriate 
knowledge and skill. 

There is a common understanding that VET-standards have to be developed according to the 
needs of the labour market (demand-led VET-standards) and not according to the supply 
(education and training system, schools, workshops etc). “The labour market” is a synonym of 
the needs of the different sectors of the economy and particularly the needs of employers 
concerning specific workplaces. Representing “the labour market” are employers, trade unions 
and government. If, as it is the case in many partner countries, employers and unions haven’t 
yet organised to articulate their needs in terms of VET-standards government has a difficult 
task: to define labour market needs which have to be met by VET-standards in such a way that 
they will be accepted by employers. 
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Most VET-systems in partner countries (and EU-member states as well) refer to “occupations” 
as the guiding principle for their VET-standardisation (entry requirements, characteristic of the 
occupation, curriculum, assessment, certification etc.). But there is no common international 
understanding of what an “occupation” is and how many there should be. The International 
Classification of Occupations (ICO) contains round about 1500. The German Vocational 
Education System names about 360 “training occupations”, while the German Employment 
Office lists some 35.000 jobs. The Russian Federation, according to it’s 1995 Education Law, 
has developed 243 training occupations, the Ukraine recently some 360. It seems to be 
necessary to focus on the difference between “training occupations, for which VET-standards 
are being developed and jobs at the workplace. 

A ’’training occupation” comprises broad basic skills and knowledge which provide the 
fundamental competencies to perform in a particular job and to be able to take up life 
long learning. 

Taking into consideration that there is no common understanding of the term “occupation” and 
after agreeing on a common definition of the term „VET-standards“ the central point of 
discussion in subgroup C was whether members could agree on a small number of types of 
standards. This step was held necessary for the future debate among countries with very 
different VET systems. Finally, three archetypes were described which seemed to be covering 
most of the possible VET-standard development in EU and partner countries. 

The Examination or Output Model represents a standard which merely describes the 
examination or qualification requirements. The preparation for such an examination is not 
described in that type of standard. There is no curriculum, no entry requirements, no courses or 
training venues mentioned. The learning process (input) is not covered by this model. There 
have to be examination bodies (chambers, training institutions etc.) that provide examination 
facilities. The UK has developed a model of assessable standards of occupational competence 
with it’s National Vocational Qualifications (NVQ) which serves as a pilot model in other 
countries like South Africa and Australia. Using subgroup C's definition the UK’s NVQ are an 
“Examination or output model” because it lacks the criteria of an “occupational model” like 
curricula, duration of training etc. 



Models of VET-Standards 



Output Model 


Professional/ 

Occupational 

Model 


Modular Model 


Elements of Standards 




X 




characteristics of the profession/occ. 




X 


X 


duration of training 




X 


X 


training objectives and content 


X 


X 


X 


examination requirements and 
procedure, certification 



The Occupational or professional Model represents the broadest standard among the three 
archetypes. Besides the obligatory elements that are shown above the standard should outline or 
recommend other important components like 






students’ entry qualification; 



• teachers’ and trainers’ qualification; 

• recommendations for the use of media and training aids; 

• recommendations for literature and methodological advice. 

The “occupational model” is used in most of the EU-VET-Systems and partner states which 
keep this model due to a long tradition in their countries. 

The Modular Model represents a type of standards that does not follow a curricular track of 
two or more years like the occupational model, but it sets assessable qualification standards for 
certain occupational activities. These can also be part of an occupational standard, but separate 
qualifications with certification. The different modules can be compiled to become a 
comprehensive occupational qualification, if all elements of a defined occupational standard 
are brought together. 

As an important outcome of the 1995 work session most members of Subgroup C were in 
favour of the occupational model. Most of the standards that have been developed within the 
last three years are professional/occupational standards. Some countries have developed 
modular standards for specialised occupational activities aside their occupational standards. 

3.2. Standard Development (1996) 

Subgroup C, after a long discussion about very different procedures and strategies for standard 
development, decided on a kind of check - list of steps to develop a standard. It was agreed that 
even if the procedures in the different countries would remain different a check-list would be a 
professional help in the context of a fundamental reform of standard development. Partner 
states were informed that there are no European terms of reference for VET-standard 
development or VET-Standards. The only EU-orientation were the five EU-levels of education, 
the EU-comparative studies provided by CEDEFOP and the EU-occupational classifications 
used by the employment offices in EU-countries. It was not looked upon to be worthwhile to 
find an agreement on a common level for skilled workers’ qualifications. A first short 
discussion showed that members ranged skilled workers’ qualifications from EU-level 2 to 4. 

Standard Development Model Scheme (check list) 

1. Commission/Application/Desires, i.e. analysis, draft, delphi; 

2. Testing/Decision, i.e. testing in schools, political decision; 

3. Team Formation, i.e. standard writing team formed by researchers, educators, employers, 
employees etc.; 

4. Development Process, i.e. draft, pilot project, evaluation, first revision; 

5. Submission for Decision, i.e. presentation of standard to decision makers/ ministry; 

6. Decision and inacting of Standard, i.e. publishing, teacher/trainer preparation, 
development of learning materials; 

7. Trial Stage, i.e. first implementation in a small number of pilot projects; 
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Large Scale Implementation, all schools and training institutions now inact the new 
standards; 

9. Evaluation, analysis of performance; 

10. Revision, in the light of the first experiences - after one or two training periods - the 
standard will be revised. 

Subgroup C has worked out a detailed check-list that was thought to serve as a first aid for 
those who were urged into a large number of standard setting processes. 

3.3. Standard Implementation (1997) 



Subgroup C’s next step was to work out strategies and aids for the implementation of standards. 
The yearly advancement of the Subgroup — 1995: definition / 1996: development / 1997: 
implementation — did, of course, not mirror the different stages and procedures of standard 
setting in the member and partner states. Most of the countries were still busy to develop their 
first standards when others already started the implementation. This will be the characteristic 
feature of VET-Standard work in the years to come in all members and partner states: parallel 
activities to define, develop, implement, evaluate and revise standards. 

In 1997 ETF launched a „Standard 2000“ pilot project which consists of a number of separate 
and consecutive activities: 

• pilot standard projects are proposed by partner states in early 1997; 

• Subgroup C discusses proposals in 1997 (Prague); 

• Phare programme experts and Subgroup C exchange experiences of curriculum work 
(Prague); 

• a Standard Development Team (SDT) is formed by Subgroup members to consult, 
monitor and evaluate pilot projects; 

• country specific projects are launched in 1997; 

• projects are to be evaluated in 1998/99; 

• results are disseminated in 1998/2000. 

The Standard Development Team (SDT) is supposed to render assistance to partner countries 
in the course of standard development and implementation and do the first steps of evaluation 
between the meetings of Subgroup C. One Tacis country (UKR), two Phare (CR, LAT) and two 
EU-countries (NL, D) formed the SDT. In the 1997 meeting of Subgroup C three countries in 
the framework of „Standard 2000“ presented their first experiences with standard 
implementation (CR, LAT, SLO), three gave a short outline of their procedures (Bulgaria, 
Lithuania, Romania). Seven countries announced to propose pilot standard projects in 1997 
(Kyrgystan, Azerbaijan, Mongolia, Russian Federation, Ukraine, Moldova and Former 
Yugoslav Republic of Macedonia). One result of the discussions so far confirmed what had 
been assumed in the first meeting of Subgroup C. 

The VET-systems of the partner states are of very different structure and their standards 
show an enormous variation. They ask ETF to provide individual advice and consultancy 
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to accommodate such variations - including both „top down" and „ bottom up" 
activities - in order to achieve at least comparable procedures and framework for VET- 

Standards 

Comparability of VET-Standards was an important issue of Subgroup C’s discussion in 
1997. Czech Republic (Chair: Mr. B. Janys) and Slovenia (Expert: Mr. V. Tkalec) proposed a 
set of indicators for comparison which was adopted by the Subgroup: 

Indicators for VET-standard comparison 

1. System Approach, i.e. school based, work and school based; 

2. Curriculum, i.e. objectives, contents, duration; 

3. Capacity of the System, i.e. schools, companies; students, teacher/trainers; 

4. Financing, who pays for what? 

5. Teachers/trainers, professional preparation, continuing education; 

6. Understanding of VET, i.e. narrow/specialised, broad/basic, key competencies, different 
vocational tracks, system or programme; 

7. Parity of Esteem, i.e. access to higher education; 

8. Productivity of the VET-System, i.e. percentage of age cohort that is entering/leaving the 
system, drop-out rate; 

9. Acceptance with Employers, i.e. student placement on graduation, demand of the labour 
market, percentage of unemployed graduates, employer involvement in standard setting. 

Phare-Programme meets Subgroup C „VET-Standards“. The exchange of experience 
between Phare- and Subgroup C-experts turned out to be very fruitful. In the course of the 
(older) Phare programme a large proportion of work was dedicated to curriculum development. 
Although the scope of projects in Phare was reaching far beyond VET-Standards there was a 
common understanding between the two expert groups on the functions of VET standards: 

1. VET-Standards should „ensure links between vocational training and the economic 
sector in which participants should find work after training, “ 

2. VET-Standards should „ensure the compatibility of different diplomas/ certificates 
within a country" , 

3. VET-Standards should „provide qualifications which are recognisable in EU-member 
states." 

(All citations from Bohumil J any s/Vladimir Tkalec „ Strategy and aids for the implementation of 
vocational standards", Prague 1997). 

In all Phare projects entry requirements have been specified but it was not evident whether all 
Phare projects pay attention to examination requirements. Phare and Standard experts agreed in 
the recommendation 

• that all countries should pay more attention to industry needs and to ensure linkages with 
the world of work by tripartite structures and research. 




13 



that structuring curricula in a modularised way could facilitate the development of 
occupational standards and the achievement of educational objectives. 



4 . Preparation of subgroup C’s 1998 topic: 
Evaluation/Progress Review 



4.1. Subgroup C’s 1998 topic: Evaluation/Progress Review of VET- 
Standards 

As a result of the 1997 Advisory Forum meeting in Turin ETF decided that the topic of 
Subgroup C’s work in 1998 would be the evaluation/review of the development and 
implementation of VET-Standards. The group was well aware that a review of implementation 
at such an early stage of the process would be able to meet the requirements of a n evaluation in 
the classic sense. Therefore the group recommended, that a thorough content related exercise of 
a standard evaluation which includes the output of VET-processes should take place at a later 
stage and should be carried out by independent teams of experts. 

Anatolij Simak (UKR) was appointed Chairman of Subgroup C in 1998 and Hermann W. 
Schmidt (D) expert to the chair. Thomas Schroder (ETF) supports the Subgroup as in the years 
before. In a chair and expert meeting in Turin in Dec. 1997 ETF-directorate and Subgroup C 
chairman agreed on the following procedure to carry out Subgroup C’s tasks: 

4.2 Evaluation/review of the process procedure 



The objective of our work in 1998 was to develop a first specific for an empirical 
evaluation/progress review of pilot projects for VET-Standards within „Standard 2000“ 
covering the following areas: 

1 . the description of the processes of standard development and implementation with regard 
to Subgroup C’s definitions, principles for the development and strategy 
recommendations for the implementation; 

2. different case analysis of 

* vocational education and training pathways, 

* identification of theoretical and practical elements of standards, 

* objectives and content of standards according to the chosen model: 

- occupational/professional model, curriculum analysis, 

- modular model, curriculum analysis, 

- output/examination model, examination analysis 

- mixed model, analysis of different objectives/needs 

* examination, functions and procedures i.e. permanent assessment, 
intermediate/final examination, achievement in terms of levels, testing procedures 
for skills, knowledge, key competencies, 

* private sector participation in 

- analysing industry/labour market needs, 

- standard development and implementation, 

* standard acceptance with 



- education and training institutions, i.e. schools, companies, workshops, 

- students and students' parents, 

- teachers/trainers, 

- the public, i.e. press; 

3. bilateral or international comparison of standards, i.e. between two partner countries, 
between one partner and one EU-country, between all partner countries with pilot 
projects. 

4.3 To be evaluated/reviewed: Competence and performance based 
VET-Standards 

All standard models, which have been discussed in Subgroup C — occupational, modular, 
output — have one characteristic in common: they are competence and /or performance based. 
That means that these standards define the competence and/or performance which is regarded 
as satisfactory for a particular occupational/professional role. Therefore, the final examination 
should pay high attention to the assessment of competence and performance.. 

The UK served as an example for performance oriented standards. With the setting of national 
output standards in the UK, the National Vocational Qualifications (NVQ), the following 
definition of standards was given: 

„ Standards are benchmarks; descriptions of the expectations of employment against 
which the actual performance of individuals will be compared and assessed as competent 
or not competent as appropriate. “ (Mansfield. R.( 1991): Deriving Standards of Competence. In: 
Fennel, E. (ed.): Development of Accessible Standards for National Certification, Department of 
Employment, Sheffield) 

Following this definition, these standards place the emphasis not on a broad field of knowing 
and understanding, but only on performance, i.e. what students are able to do. Therefore 
standards are also expected to provide points of reference for the assessment of performance.Of 
course, there is criticism of such standards because there is no such thing like an absolute and 
objectively defined occupational role. Many car-mechanics would not agree with their job-role 
as it is described within a VET-standard. Because of this notion it is absolutely necessary to 
include researchers and as many practitioners as possible in the course of standard development 
in order to achieve a broad acceptance. For the evaluation process referring to practitioners’ 
involvement is helpful as well. 



4.4. Questionnaire „Evaluating /Reviewing Standards" 



One questionnaire has been circulated among members of the subgroup in January 98 to 
evaluate the procedure of standard development and implementation in member and partner 
countries. The questionnaire follows results of Subgroup C’s work starting with the models of 
standards, the development steps and the implementation procedure. The Standard 
Development Team agreed that the questionnaire should be short, in order to get as many 
answers as possible. 



Nine out of fourteen partner states and six EU-member states answered the questionnaire, the 
results of which are shown in par. 5.3.2. The main reason for six partner states not to answer 
the questionnaire was the early stage of their VET-standard development work. These countries 
are still preparing VET- standards. 

4.5. Key questions 

In the course of Subgroup C’s work many questions have been asked concerning the 
appropriate understanding of defining, setting and implementing modem VET-standards. The 
Subgroup has tried to develop a common methodology of „typical“ procedures well knowing 
that this is a recommendation which has to be altered if the circumstances in a particular 
country make this necessary. Some questions have emerged from that debate to be key 
questions which are arising everywhere and the answers to which can help others to proceed. 
Some of these questions are listed below. 

The following questions should be subject to discussion in order to develop a set of key 
questions that can be used in the evaluation/review process: 

1. How have Subgroup C’s recommendations for standard setting been followed and 
applied/changed? 

2. What were the reasons to change the procedure? 

3. What different steps were taken? 

4. How were labour market needs/company desires taken in account in the development 
process? 

5. Has labour market and vocational education research been involved? What was it’s 
contribution? 

6. How have teachers/trainers been prepared for standard implementation? 

7. How have the standards been accepted by schools, companies, trade unions, 
teachers/trainers, students, the labour market, parents, the public? 

8. How should Subgroup C’s recommendations be revised? 

9. Where are we now in this process? 

10. What do we propose for Subgroup C’s future work? 



5 . 



The evaluation /progress review of VET-Standard 

DEVELOPMENT AND IMPLEMENTATION 



5.1. Present conditions of VET-standards in partner states 

In order to adequately judge the very different situations in the countries of the former eastern 
bloc one must take in account, that some countries, mainly those in 

• central and eastern Europe had well developed VET-systems before World War II, while 
others, mainly the 

• New Independent States did not develop VET-systems of their own but were dependent 
on the central VET policy of the Soviet Union, its standards, research in methodology, 
textbooks etc. 

But even the former eastern bloc members of central Europe have enormous problems to 
transform their VET-systems. As one member of Subgroup C puts it: “Since two years we are 
— in the frame of this group’s work on standards — leading a discussion about how to set 
VET-standards, but until now we have only prepared a first draft. We are not used to have so 
many different interests to participate. We want to be up-to-date, but we also need to be 
realistic. Because of the very different opinions and situations in the different branches of the 
economy we have not yet a clear answer to many difficult questions as we have not yet tested 
how a new system of VET-standards works." 

Although these countries have VET research capacity — which had been built up in the SU and 
those central European countries who belonged to the eastern bloc in the fifties and sixties of 
this century — they are struggling with problems that are typical for all partner states 

• their VET - systems are mainly school based; the former infrastructure of public 
enterprises which were responsible for practical training in co-operation with VET- 
schools and for employing the graduates of these schools is broken apart with some 
relicts here and there were co-operation is still functioning or were it has already been 
rebuilt. 

• A social partnership (employer - trade union- relation) as it is known in many European 
countries does not exist yet. First contacts have been built up with companies or 
employer organisations, but very little to none with work councils in companies or trade 
union representatives. 

• The Russian Federation and the countries in central Europe (not the Baltic States) have a 
good VET-research and development infrastructure, which they can rely on when 
educational problems are concerned, i.e. teacher training, educational materials, 
curricula. 



• Nevertheless, the preconditions for quick and valid labour market analysis are not the 
best, because of the lack of experienced labour market administration, employment 
offices and labour market research under conditions of a free labour market. 

• General education has a long tradition and plays a major role in VET and, of course, in 
VET-standards. It was and is part of the educational philosophy in partner countries that 
the main objective of all education, general and vocational, is the development of the 
human personality. In addition to this educational policy issue these countries assume 
(by many good reasons) that major elements of general education like language, 
mathematics and basic knowledge of science become more and more important in all 
occupational and professional learning processes (especially in information and 
communication technology) and therefore should be a significant part of VET-curricula. 
The proportion of general education in many partner states equals 50 % of the total 
curriculum. These countries now face a problem which is well known to western VET- 
standard setters. From the point of view of an employer such a high proportion of general 
education is not necessary to be trained to a good occupational standard, although there 
is permanent complaint of trainers about deficiencies in reading, writing and 
communication skills with many trainees. As it comes to the costs which undoubtedly 
will be higher with such a high proportion of general education there is little hope to 
educate all young people to these ambitious standards even for those governments which 
pursue these educational objectives. 

• There is little experience with market economy occupations like business and 
management occupations, banking, insurance, international trade which are to be 
introduced into VET-systems dominated by technical education. The problem is not to 
develop such standards (with ETF consultation) but to train teachers and trainers to these 
standards and to implement the standards in schools and workshops or companies. 

5.2. ETF - Responses to partner state needs 

ETF responded to these different needs of partner states by 

• establishing information and data collecting agencies, so-called observatories, which 
among other data collect and provide information about the preconditions of vocational 
education and training like labour market needs; 

• developing an expert network with procedural recommendations for the definition, 
development, implementation and evaluation of VET-standards through Advisory Forum 
members in Subgroup C; 

• launching and monitoring pilot projects in partner countries; 

• providing tailor made advice for countries which start to develop VET- standards for the 
first time; 

• implementing an expert information network between partner and EU- member states 
through special publications. 



5.3. The evaluation/review of implementation of VET -Standards 



As mentioned earlier the “evaluation” as the topic of subgroup C’s work in 1998 was meant to 
be a first review of the development and implementation process at a very early stage. All 
partner states, including those that are still preparing to develop VET-standards were interested 
to learn from others how they coped with problems of implementation. The concern of this 
evaluation/review was from the very beginning focussing on the process of implementation. 
Output and results of education and training under the new standards, assessment and 
certification will be subject to evaluation in the years to come. Then the classic criteria of 
evaluation will have to be applied: effectiveness (were the objectives achieved?), impact (has 
the implementation changed anything?), relevance (were the objectives meaningful in terms of 
labour market needs?), efficiency (were the resources invested rightly spent?) and 
sustainability. 

5.3.1. What is to do - what can be done? 

The evaluation/review of VET-standards is the topic of Subgroup C’s 1998 work 

programme and was intensively discussed in the Kiev meeting. There were four 

questions identified that have to be answered in the course of evaluation/review: 

* Why do we evaluate VET-Standards? 

* Who evaluates? 

* What do we evaluate? 

* How do we evaluate? 

a) Why do we evaluate/review VET-Standards? 

VET- standards usually reflect the technologies, work organisations, customer 
orientation, the position of the national economy in global markets, 
internationalisation of companies, the labour market situation (national, regional, 
local), job classifications, emerging markets (for instance: multimedia) and other 
factors in the world of work. On the other hand, VET as a part of the national 
education system reflects the guidelines of the national VET policy. Thus, the time 
to evaluate VET-standards has come, when one or more of the above mentioned 
parameters change and a VET reform is launched. While EU-member states are 
evaluating their VET-standards because of the overall changes in the economy 
with a modernisation of existing standards and the creation of new occupations, 
the partner states face a radical reform of their education systems, their economies 
and, respectively, their labour markets, job classifications and all educational 
standards, including VET. The answer to the “Why” is to safeguard an overall 
education reform process that just has begun. 
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Who evaluates/reviews? 

The most important rule of a valid evaluation is to nominate independent, 
knowledgeable professionals for the evaluation team who are not linked with the 
development or implementation of VET-standards. It is agreed that it is difficult to 
join independence and knowledge, especially within the ongoing process of 
dynamic changes and strong political pressure on the education system to hurry up 
with reform. Those who develop new standards and implement them usually are 
the most knowledgeable people in this field until the innovation has settled into an 
overall practice. One of the keys to solve that problem is to involve VET research 
from the very beginning of the development process. 

What do we evaluate/review? 

Of course, there is a very broad spectrum of factors that can be subject to VET- 
standard evaluation but rather at the end of the process of implementation. One 
could differentiate three fields of evaluation/review for our current work: 

- the initiation of the process of standard setting, 

- the development of VET-standards, 

- the implementation of VET-standards, 
and - at a later stage of the implementation - 

- the effects of VET-standards with individuals, in educational 
establishments, in the labour market, personnel development in companies, 
etc. 

As before mentioned the initiative for standardisation can already now be subject 
to evaluation. Who takes the initiative or who has the right to take such an 
initiative. Who nominates the evaluators? What interests are involved with the 
nominators and the nominees? What is subject to and the method of evaluation? 
Has all this been made public and known to the professional and scientific 
community? 

In order to validly evaluate the development process it has to be made clear what 
components the VET-standard will consist of because different evaluation 
methods and standards have to be developed for these components as there are 
among others 

- access requirements for that specific occupational track (is it open to all 
school leavers or does the applicant have to have a specific qualification?), 

- occupation or job characteristics (main functions or tasks of that specific 
occupation), 

- curriculum (core curriculum with additional eligible elements, method 
recommendations, teaching materials etc.), 

- examination requirements (intermediate, final) and examination procedure 
(verbal, written, task performance etc.), 

- teacher/trainer qualification and further education and training, especially 
the introduction into the new standards, 

- education and training venues (schools, workshops, laboratories; companies 
etc.), 



the quality of the education and training process (Input quality measuring as 
described in the norm of the International Standard Organisation ISO 9001 
or -much more difficult - output quality measuring as in the final 
examination and by judgement of people who employ graduates; therefore, 
follow up studies are needed). 

These and other items can be subject of the evaluation during the development and 
implementation process. In addition to that the procedure of evaluation itself 
should be evaluated, too. This concerns the question. 

d) How do we evaluate / review? 

Test the reaction of those who are concerned 

- the individual learner (the student, the unemployed in further education, the 
worker who wants to be upgraded etc.), 

- the teacher/trainer who has to teach/train to the new standards, 

- parents who are interested in a qualitative education of their children, 
the company that is involved in the training process, 

- the employer who hires a graduate? 

Although it is more expensive and time consuming than working with 
questionnaire interviews should be conducted in order to take advantage of the 
personal contact with those stakeholders in the process. The results of such 
evaluation studies should be discussed with some the people who have been 
interviewed and be published. 

One of the most difficult tasks of evaluation is the measuring of the quality of the 
education process itself. Introducing ISO 9001 as an international norm to evaluate 
the performance of educational establishments does not ensure a high output 
quality because it evaluates the standard of the educational process, but this 
evaluation proves whether the educational process meets the state of the art. 

When the Chair of Subgroup C, the expert to the chair and ETF decided in 1997 to 
circulate a questionnaire on VET-standard-evaluation among the members of the 
Subgroup they knew that the standard setting process was at very different stages 
in partner and EU-member states. Taking in account that there was a very short 
time frame before the next meeting they formulated a short questionnaire which 
was thought to disclose first experiences from that very interesting process of 
standardisation. The following chapter shows the results. 

5.3.2 Results of the 1998 questionnaire on VET-standard evaluation/progress 
review 

Again it must be stated clearly that the emphasis of this questionnaire at this stage of the 
process of VET-standard development and implementation was put on the progress that 
partner and member states have made in the transformation/modernisation of their VET 
- standards. It is to early to investigate on the impact of VET-standards on the labour, on 
companies' strategy to hire people or on individual careers. 

Questions and Answers 
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a) Which VET system is typical for your country? 



Answers 


Partner 

countries 


Member 

states 


All 


• school/theory & practice / work 
experience in companies 
(internship) 


8 (89%) 


6(100%) 


14(94%) 


• school with theory & practice 


2(21%) 


2 (33%) 


4 (27%) 


• school (only theory) 


- 


- 


- 



School based VET systems are prevailing in partner and EU-countries. They 
mostly comprise classroom teaching (theory) with practical demonstrations 
(teacher centred) and the performance of work tasks (learner centred) in 
workshops, cabinets and laboratories. Although dual education and training 
systems do not exist in partner countries, as they do in some EU member states 
(i.e. Austria, Denmark, Germany, Netherlands), most VET systems include 
practical work experience in private or public enterprises (internships). 

b) Which VET standard model did you choose? 



Answers 


Partner 

countries 


Member states 


All 


a) the “ occupation model” 


3 (33%) 


3 (50%) 


6 (40%) 


b) the “module model” 


- 


- 


- 


c) the “ output model” 


1 (11%) 


- 


1 (7%) 


d) a “mixed model” 

• (a) + (b) 

• (a) + (c) 

• “ decentralised model” 


4 (45%) 
1 (11%) 


2 (33%) 
1 (17%) 


6 (40%) 
1 (7%) 

1 (7%) 
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The traditional VET standard model in Europe — east and west — is the 
“occupation model” which can also be described as the “education track model” 
or as the “ input-process-output model” . It comprises obligatory elements like 

- access requirements, 

- occupation characteristics, 

- an education and training curriculum with a time frame, learning objectives, 
the description of contents and tasks to be preferred (input) in the learning 
process, 

- examination requirements (output), 

and voluntary elements and recommendations like 

- teacher/trainer education and training, 

- methodological repertoire, 

- education aids/material, multimedia, 

- minimum equipment of workshops, etc. 

Although most partner and member states name the “occupation model” 
traditional or inherited they have decided to keep the model but modernise it, 
restructure and broaden their occupation concept and set new VET standards. For 
partner states this decision means much more than modernisation of existing 
occupational standards. They have to develop new occupational classifications that 
serve the needs of a market economy and individual expectations for employment. 
Since the “occupation model” needs a time frame of three to five years to react 
upon fundamental changes in technology and work organisation they flexibilise 
their “occupation model” by introducing a “module model” in addition to the 
occupational model. This model combines a syllabus of performance based 
learning objectives with examination requirements for short term (2 months to one 
year) training periods and qualifications. These modules can be put together to a 
broad qualification. Also some EU members state that the “occupation model” 
being mixed with the “modular model” makes their VET systems more flexible 
and is used to respond to short term labour market needs. On the other hand the 
“modular model” lacks the advantages of the “occupation model”: individual 
identification with the occupation or profession which gives status in society, 
ensures a specific income in the frame of collective bargaining agreements, 
ensures a specific level of social security and permits to market ones qualification 
on regional or national labour markets were the "occupation model" is practised. 



Who took the initiative for VET standard setting? 

It is the ministry of education which takes this initiative in 2/3 of partner and EU 
states. Other ministries (labour, economy), schools or research institutes have also 
taken the first step. Social partners take the initiative only in some EU countries 
but not in partner states. Their role as a partner for government in VET planning 
processes has to be developed yet. 

What were your steps in the development process? 



8 steps had been described by ETF Subgroup C in 1996 as a kind of check list for 
VET standard development. 





Partner countries 


Member states 


All 


1) demand analysis 


6 


6 


12 


2) international comparison 


8 


3 


11 


3) bench marking to 
international standards 


6 


3 


9 


4) deficit analysis of existing 
standards 


6 


3 


9 


5) deficit analysis of new 
standards 


8 


5 


13 


6) development of 1 st draft 


9 


6 


15 


7) experts discussions 


7 


4 


11 


8) revision, 2 nd draft 


1 




1 



Obviously partner states used the check list as a procedural recommendation for 
new paths in standard setting whereas EU member states more often followed 
other, more traditional ways. 



What do you do while implementing standards? 
=> Evaluate the acceptance with 





Partner countries 


Member states 


All 


• teachers 


9 (100%) 


4 (67%) 


13 (87%) 


• students 


4 (45%) 


1 (17%) 


5 (34%) 


• parents 


2 (14%) 


1 (17%) 


3 (20%) 


• companies 


8 (89%) 


- 


8 (60%) 


• trade unions 


2 (22%) 


5 (84%) 


7 (47%) 


• employers 
organisations 


6 (67%) 


5 (84%) 


1 1 (74%) 


• labour market 


7 (78%) 


2 (34%) 


9 (60%) 



Partner states test the acceptance of new VET standards primarily with their 
schools, with teachers and students, and local companies while EU countries 
practice their tripartite VET partnership with employer organisations and trade 
unions to assess new VET-standards. “The labour market”, usually represented by 
the department of labour and employment offices, is an important partner for the 
assessment of new VET standard for seven out of nine partner states while in the 
EU this is only the case with 2 out of 6. 

=> Preparation for education and training 

Two third of all members in east and west develop teaching materials along with 
VET standards. These aids are looked upon as an effective tool for 
implementation. 

All countries name teacher training as the most important factor for successful 
implementation, but due to time pressure more than 90% train teachers in the 
course of introducing standards and not before. 

Only one third of members revise and work a second draft standard before 
implementation. 

When does the first revision take place? 

The lack of experience may be the reason why the answers vary between one and 
five years with most answers (4) on the five year term. One EU member revises 
“ on request” . 

=> What is the duration of a VET curriculum? 



Answer 


Partner 

Countries 


Member states 


All 


• One year 


2 (22%) 


1 (17%) 


3 (20%) 


• Two years 


2 (22%) 


3 (50%) 


5 (34%) 


• Three years 


6 (67%) 


5 (84%) 


1 1 (74%) 


• More than three years 


5 (56%) 


3 (50%) 


8 (54%) 



All countries have VET tracks of different duration, but a significant proportion of 
more than 2/3 offer long term VET tracks of three and more years. 



g) Did you refer to the five categories of VET-standards which Subgroup C has 

defined? 



Answers 


Partner countries 


Member states 


All 


• access requirements 


9(100%) 


2 (34%) 


1 1 (74%) 


• curriculum 


9(100%) 


3 (50%) 


12 (80%) 


• examination requirements 
and procedure 


9 (100%) 


3 (50%) 


12 (80%) 


• educational tools and media 


8 (89%) 


3 (50%) 


1 1 (74%) 


• teacher training 


9(100%) 


3 (50%) 


12 (80%) 



All countries have proved to take these 5 categories into consideration in the 
process of setting and implementing new VET standards. 3 EU member states did 
not fill in answers to these questions at all, but in discussions they affirmed that 
these categories also apply to their understanding of VET-standards. 
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6. Presentation of current activities in 4 countries: 
Russian Federation, Ukraine, Former Yugoslav 
Republic of Macedonian and Germany 

While planning the 1998 meeting of Subgroup C ETF and the Standard Development Team 
originally thought of the presentation of four pilot projects of VET-Standards in the four 
countries. Later on it seemed to be more interesting to hear about the overall VET reform 
policy in the Russian Federation and Ukraine following the recent basic educational legislation 
which sets the frame for a new understanding of VET-standards in these countries. The former 
Yugoslavia Republic of Macedonian introduces an interesting method of standardisation with a 
focus on quality assurance and Germany presents VET-standard-setting procedures and 
standards in an emerging labour market (information and communication technology). 



6.1. Russian Federation (RF) 

The process of VET-standard setting in the RF has reached a new dimension after the “Russian 
Federal law on Education” from 1996 was enacted. 

This law introduced “ State education standards” for all sectors of education, especially for four 
levels in the field of VET (article 9) 

1. primary vocational education; 

2. medium vocational education; 

3. higher vocational education; 

4. post-graduate vocational education. 

The responsibility of the Russian Federal Government is to 

• establish an occupational classification; 

• establish federal elements of state education standards; and 

• ensure the federal elements in state education standards. 

This is a very new issue of federal policy because for the first time the central government 

• sets a core national VET-standard which gives room to the regions and educational 
establishments to set additional standards to their needs “within the boundaries 
established by the laws of the RF“ , and 

• gives educational establishments the responsibility to draw up and approve educational 
programmes and curricula. 
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The ministry of education has developed VET standards (“federal elements”) for 263 
occupations which are to be finished and completed within the regions. Most occupational 
standards are in the development and implementation process. 

6.2. Ukraine (UKR) 

The new Ukrainian Vocational Education Act (UVEA) has recently passed parliament with all 
political parties backing the overall VET concept, but also especially the Ukrainian concept of 
VET standards: 

“A state standard of vocational education" ("vocational and technical education”) 
contains the overall state requirements concerning the contents of training, the 
qualification level, ..., the fundamental compulsory training resources and the 
educational level of the graduate. According to this, state standards are developed for 
each occupation which is contained in the “ inventory of occupations for the training, of 
skilled workers in vocational schools (UVEA, article 33) 

In the meantime standards have been set up or are in the process to be set up for round about 
240 occupations. Ukrainian VET standards comprise 

• an occupational profile; 

• an occupational framework syllabus; 

• framework syllabi for subjects and practical training; 

• examination requirements and performance criteria; 

• a list of binding education and training tools; 

• requirements for the educational level of the school leaving certificate; 

• qualification level of teachers/trainers. 

The VET standard for “office secretariat referent” which has been developed as a pilot project 
in the frame of the ETF sponsored “ Standard 2000” programme is one of the first standards 
that meets the new legal criteria. 



6.3. Former Yugoslav Republic of Macedonian (FYRM) 



VET standard development is part of the overall VET reform in FYROM. With support of ETF 
pilot project “ Standard 2000” a standard development team has designed a concept for VET 
standards and developed five VET standards for the hotel, restaurant and catering industry that 
are implemented in 1998. 
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The philosophy underlying the FYROM concept takes four parameters into consideration in 
order to develop VET standards: 

• labour market; 

• education policy; 

• personal needs of the individual; and 

• educational institutions. 

Job description and job design are mirrored in national labour market needs. These factors 
determine widely the new VET standards. Curriculum development for specific occupational 
and new VET school organisation and methodology are to ensure the quality of delivery. Thus, 
the VET reform is governed by a set of networking measures within this triangle. 



VET-STANDARDS 












VET QUALITY 





VET CURRICULA 





The FYRM pilot project introduced a well elaborated input-performance-output model for VET 
standard development and implementation consisting of eight components which gave a clear 
inside view on the FYROM laboratory work of VET reform. 

A very interesting feature of the FYRM project is the introduction of quality measurement for 
the educational process by using the norm ISO 9001 of the International Standard Organisation 
which is used in many countries to safeguard quality in education. I t will be of major 
importance for the future procedure in quality assurance to hear from FYRM about the results 
of this kind of evaluation. 



6.4. Germany 



The development and implementation of the standards for four information technology (IT) 
occupations in Germany within the last two years were presented as an example for the reform 
of the standard setting process and the occupational classification within the dual system of 
education and training in Germany: 

• IT electronics technician; 

• IT system support specialists; 

• IT specialist; 

• IT officer. 

Not only these occupations are new (and not just re-named older occupations) but the 
mechanism of developing and implementing the new standards has changed tremendously. 
Government, employer organisations, trade unions and VET research are under heavy pressure 
of the public opinion and enterprises to develop new VET occupations according to the new 
needs, with global players, but also fast growing small businesses in emerging markets like IT, 
multi-media, microtechnology and all kinds of services. 

The initiative to develop these new IT-occupations was taken by the employer organisation of 
the electronic industry and the metal workers’ union. In a tripartite team, together with research 
support from Federal Institute for Vocational Training, the four standards were developed 
within nine months. It took 16 months from the initiative to the beginning of training on 1 
August 1997, a record in such a planning process. Although companies and schools complained 
about the short preparation period round about 1,000 companies concluded some 4,200 training 
contracts and some 400 VET schools offered education for these trainees in 1997. Some 20.000 
additional training places are expected by employers until the year 2000. 

In the course of the reform more than 150 standards have been modernised and some 25 
standards for new occupations have been introduced into the system. 
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7. Key question: How to measure labour market 

NEEDS? 

The meeting of ETF Subgroup C “VET Standards” in Kiev brought a vivid and fruitful 
discussion on crucial points which arise in the course of standard setting and implementation. 
In an interesting dialogue with ETF Director Peter de Rooij all members of the Subgroup stated 
that 

“ETF Subgroup C’s work like recommendations, concepts and proposals are undismisable for 
their work on VET standards at home.” 

As one of the most difficult questions in the beginning of a VET standard-setting process group 
members identified the following problem: 

How are labour market needs measured to be long-term, broadly based in the economy 
and not a local or particular company need and - speaking in terms of occupational 
skills and knowledge, attitudes and capabilities - broad enough to fill at least a two to 
three year curriculum, if you want to meet the requirements of level three in the EU set of 
educational levels (unskilled; semi-skilled; skilled worker/employee; technician; 
academic profession)? 

Different ways to measure labour market needs were mentioned but not thoroughly discussed. 
Just to keep in mind what criteria and facts have been touched on in Kiev some of the main 
points are listed in the following: 



7.1. Partners for VET planners and educators in the labour market 



• Ministry of Labour, employment offices and similar agencies: 

statistics on employment and unemployment in the different sectors of the economy and 
within different occupations; structure of unemployment (unskilled/skilled, etc.), demand 
for skilled personnel in the different sectors of the economy etc. 

• Employer organisations, trade unions, companies: 

skill needs from an employer’s or unionist’s point of view; trends of employment on 
different levels (skilled/unskilled); future needs - union/employer contribution to future 
job descriptions and workers'/employees' roles, job designs in emerging markets like 
multimedia, genetic technology, information technology etc. 

• Labour market and VET research 

analysis of labour market structure; local or regional labour market needs; statistical 
comparisons, international comparisons; skill bench marking - national and international; 
ranking of labour market needs on different levels of education, emerging markets etc. 
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7.2. Tracing needs for new occupations in emerging labour markets 



Information and communication, multimedia, microtechnology-industries, foreign languages, 
customer orientation, different service industries, especially health care, belong to the most 
dynamic sectors of the economy and labour markets with a high potential for new VET- 
standards. 

In order to establish an “ early-bird-system” for VET-standard- initiatives one could have 

• analysis of job advertisements (offer and demand); 

• analysis of further education and retraining measures 

* in companies (because of changes in work organisation, etc.), 

* with chambers and employer organisations (regional and sectoral needs), with 
private education and training providers (where there is a market, there is a need); 

• analysis of job changes of individuals within the workforce; 

• hiring practice s of “young” companies in new emerging markets etc. 



8. Recommendations for subgroup C ’ s 1 999 agenda 



As indicated in par. 7 the most difficult problem in all countries is how to measure labour 
market needs when there is a lack of experienced partners like employer organisations and 
trade unions which are able to define clearly to the education sector what their skill needs are. 
Even in EU-member states with a long tradition of social partnership problems arise when it 
comes to describe the skill needs in emerging labour markets like information technology, 
multimedia etc. For partner states the problem is much more complicated because research 
mechanisms that were used to rely on totally different economic and labour market systems are 
still adapting to the new structures of a market driven economy and labour market which are 
still in a transformation process themselves. In this unique historic development there are very 
different answers to the question how to prepare the young workforce best for a future 
economy and labour market. How to build a network of reliable employers and unionists to co- 
operate with in designing new VET-Standards? How to establish an expert network to find out 
about changes in the world of work that have to be taken into consideration while designing a 
curriculum? How to develop a labour market analysis for VET-standard purposes? 

These and other questions have to be answered in the coming years in order to develop and 
adapt VET-standards according to the changes in technology and in the economy, the changes 
in society and in learning attitudes of the young generation and many other factors that have to 
be taken into consideration in this process. 

Subgroup C proposes for it’s 1999 work period the topic “What Impact has the Labour 
Market on VET-Standards and what are the consequences for Vocational Education and 
Training?” 
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